Storages: Enhance storage logging details#10951
Conversation
Signed-off-by: JaySon-Huang <tshent@qq.com>
Signed-off-by: JaySon-Huang <tshent@qq.com>
Signed-off-by: JaySon-Huang <tshent@qq.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughThis PR adds partition-table-scan metadata and richer DeltaMerge/FileCache logging, disables one flaky KVStore test, and renames several next-gen test image and branch defaults from ChangesLogging and diagnostics enhancements
Next-gen test image and branch tag updates
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: JaySon-Huang <tshent@qq.com>
|
/retest |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: JinheLin, Lloyd-Pottiger The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
What problem does this PR solve?
Issue Number: close #10950
Problem Summary:
When diagnosing TiFlash storage and query issues in production, existing logs and execution summaries lack enough context to distinguish scan types and understand slow storage operations:
TableScanorPartitionTableScan.gtest_kvstore_fast_add_peeris flaky in CI and FAP is not planned for short-term support.What is changed and how it works?
TableScan statistics
is_partition_table_scaninTableScanStatisticsconstructor viaexecutor->has_partition_table_scan()."is_partition_table_scan": true/falseinappendExtraJson, so execution summary / slow query analysis can tell partition scans apart.DeltaMerge storage logging
SplitIngestLogContextiningestDTFilesUsingSplit:LOG_IMPLto emit begin/finish logs at INFO when elapsed > 10s, otherwise DEBUG.elapsed_secondsto per-file ingest attempt logs (INFO).reasoninsegmentSplitandsegmentMergeDeltafinish logs viamagic_enum::enum_name(reason).Segment::info()withstable_colsanddmf_disk_bytes.DMFile::getNumColumns()andStableValueSpace::getDMFilesNumColumns().S3 FileCache
FileCache::DownloadType(Foreground/Background).downloadImplsuccess log for easier disagg read path analysis.Tests
gtest_kvstore_fast_add_peer.cppwith#if 0and document why (flaky CI, FAP not planned for short-term support).Check List
Tests
Side effects
Documentation
Release note
Summary by CodeRabbit